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ABSTRACT 

The intent of this study was to investigate developmental pro= 
cedures for producing a dynamic decision model for an ongoing learning 
situation presented via computer-assisted instruction (CAI). 

In addition to student performance on learning frames, criterion 
frames and quiz items, this study was concerned with additional learning 
variables such as latencies for each of the above measures, and the 
subject's confidence of his response on the criterion and quiz questions. 
The first step involved the investigation, through correlational analysis, 
of the relationship of these variables with quiz and final examination per= 
formance for a two hour course on concepts of Boolean algebra.^ Based on 
the results of correlation and regression analyses with the quizzes and the 
final exam as the dependent measures, the relevant variables were incor- 
porated into the decision model. The decision model was then implemented 
within the training materials and validated with an independent group of 
students „ 

The adaptive decision model was shown to be effective in identi- 
fying those trainees who needed remedial instruction. The use of the 
adaptive model resulted in a number of guides which, when applied to 
future research efforts in the area of adaptive training, should result 
in significant enhancement of the learning process. 
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SECTION I 



INTRODUCTION 



A major technological advance has occurred in recent years and 
can be described as "computer technology." Many investigators (Dick, 
19i55| Na'nsen, 1966; Gentile, 1967; Zinn, 1967) report that one of the 
major advantages of using a computer for training stems from its capa- 
bility to adapt the instruction based on the individual's most recent 
performance as well as his past history. Therefore, it can be hypothe- 
sized that a computer, because of its unique monitoring capabili^, 
would be an ideal manager of an adaptive training system. "Adaptive" 
is used thjroughout this report to refer to a process or technique for 
improving performance by modifying the instructional program to meet 
the individual characteristics of the trainee. Adaptive then is synon- 
omous with individualization rather than adaptive in the constant error 
spnse. 



The>^ have been several investigations concerned with developing 
a methodology for designing and implementing adaptive training systems. 

One quantitative approach to developing an adaptive training system has 
utilized the techniques developed within mathematical learnins theory. 

The investigators attempted to utilize formal matheraaticaV^mpdels of the 
learning process as a basis for the design of adaptive training systems 
(Deer i Atkinson, 1962; Atkinson & Hansen, 1966; Groen & Atkinson, 1966; 
Karush & Deer, 1966). Deer and others (1965) concluded that "the improve- 
ment of teaching procedures by seeking an optimal way of presenting the 
stimulus materials may have important practical consequences only in 
specific situations, for example, where there is a considerable degree 
of interdependent relations among the stimulus materials." Another 
quantitative approach can be identified in the research efforts of 
Smallwood, Weinstein, and Eckles (1967). These investigators proposed 
the use of response probability estimation models to optimize the instruc- 
tional strategy. Under the probability estimation model, the ^particular 
instructional block that a trainee receives is based on decision rules 
that are dynamic in the sense that the trainee's accumulating performance 
history- affects the decision process. Thus far, progress has been dif- 
fi quit , in lising quantitative methods for adapting instruction to the 
indivi dual" differences of the learners. 

A second major techr:ique for designing adaptive systems (Cronbach, 
1967) is to adapt the instr‘^tion to various facets of the trainee's 
aptitudps or traits. However, as Cronbach & Snow (1969) have reported, 
research. in the area of aptitude- treatment interaction has produced 
inconsistent findings. In addition, adapting instruction on the basis of 
pre-conditions does not permit the flexibility to change the instructional 
program during the acquisition of the material. 
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A third major approach in designing adaptive training systems has 
involved the use of empirical techniques Data on wi thin-course variables 
generated by an initial group of trainees is utilized in establishing the 
branching or adaptation points in the instructional program, as well as 
the decision criteria to be employed at each point. As subsequent 
trainees proceed through the instructional materials, data is collected 
on the relevant variables and the trainees are branched to specific con- 
tent based on their individually generated wi thin-course performance. 

The use of this technique can be seen in the efforts of Silberman and 
others (1961). Using a short course on logic, Silberman et al. estab- 
lished branching procedures based on cumulative errors within a topic, 
but found no significant differences on criterion performance when com- 
pared with a fixed sequence. The investigators reported that "it may 
be conjectured that some measures such as response latency or subject's 
self-evaluation are more appropriate than error rate and that the com- 
puter should have considered these behavior measures for its branching 
decisions instead of, or in addition to, errors." Coulson and others 
(1962) used the logic materials as did Silberman, but their branching 
decisions were based on both the cumulative errors and the subject's 
evaluation of his own readiness to advance to new topics. Coulson et al. 
found a significant difference on criterion scores for the adaptive 
branching group when compared with a linear sequence group. 

Using empirically developed branching! rules , Mel aragno (1966) com- 
pared the effectiveness of three adaptation procedures. The first used 
previous students' performances on the training materials as^the basis for 
branching decisions. The second procedure was based on previous students' 
pre-training abilities, a prediction condition, and the third treatment, the 
linear sequence condition served as a control group. Multivariate analysis 
of posttest scores and training times indicated that the^branching con- 
dition was superior to the linear condition, but no significant dif- 
ference was found between the branching and prediction condition. 

The goal , of an adaptive training system is to optimize performance 
and/or time by adapting the instruction on the basis of each individual's 
performance. Given the unique monitoring capabilities of the computer, 
a viable approach would be to develop a decision model to be used in the 
logic of the instructional program to take into account the trainee's 
performance within the instruction and to branch him to specific con- 
tent based on that performance, in essence, an empirical approach. ^The 
question then arises as to the methodology that should be employed in 
developing such a decision model. It is to this question that-the 
statement of the problem is addressed. 



SECTION II 



STATEMENT OF THE PROBLEM 



The basic objective of this investigation was to develop a 
methodology for providing adaptive training via computer-assisted 
instruction. An effective utilization of the capabilities of the com- 
puter would involve the development of a dynamic decision model for 
adapting training, The implications for the development of a dynamic 
adaptive decision model would be the use of performance data collected 
during the actual presentation of the instructional materials to modify 
the trainee's performance within the course of training as well as 
future trainees. 

The development of the adaptive decision model thus involves 
the identification of variables to be monitored during learning and the 
establishment of decision criteria for applying these variables to 
modify the behavior towards the training objectives, The use of 
regression analysis techniques would be an appropriate basis for ^build- 
ing the adaptive model because variables that are effective predictors 
of final performance would be identified, and the relative weighting 
factors produced could be used in the decision process. This empirical 
approach for regression techniques utilizing with in-course performance 
data involves a two-step process. An initial group of trainees would 
proceed through the training materials. Their^data would be analyzed 
by regression analysis, and on the basis of this analysis, the adaptive 
decision model would be developed. Although linear regression has been 
extensively used in the analysis of data, it has yet to be utilized in 
developing dynamic decision models for adaptive training. The rationale 
for this empirical approach is that the performance data collected from 
the trainees during the actual training activity would more clearly 
reflect both the types of variables to be used and the ^ types of decisions 
that are required to develop an effective adaptive training system. 

The initial problem involved the identification of potential pre- 
dictor variables that could be measured during learning, and that would 
enhance the effectiveness of -the decision model. Certainly the most 
obvious learning variable which could be measured was the probability of 
correct responding. It was hypothesized that the greater the frequency 
of correct responding, the greater the probability that the trainees 
would also perform successfully on the final examination. A second 
variable identified for inclusion in the decision model was response 
latency. Latency is defined as the interval between the presentation 
of an information-question unit to a trainee and his response to that 
unit. It was hypothesized that since latency reflects varying degrees 
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of facility with the instryctional materials, the inclusion of this 
variable should add to the predictive capability of the model. A third 
variable, a performance related variable, considered for inclusion in 
the adaptive model was subjective confidence. Previous research by 
Shuford and Massengill (1966) and others has indicated that additional 
information can be gained from subjective confidence measures which is 
not available from conventional right-wrong scoring of test information. 
Numerous suggestions appear in the writing of Shuford and Massengill 
that confidence ratings can be used not only In testing situations^ but 
in an ongoing learning environment. Therefore, it can be hypothesized 
that the utilization of subjective confidence, i.e., an indication by^ 
the trainee of his confidence in his progress with the J earning materials, 
might increase the effectiveness of the adaptive decision model. 

The second major problem area concerns the determination of the 
unit of analysis. That is, would the most effective adaptation be obtained 
by analyzing the learning data over a series of frames covering a specific 
concept, or should the data be analyzed over a series of related ^concepts 
covering a larger number of frames? The establishment of the unit of 
analysis determines the point at which adaptation can be effected. The 
unit of analysis should be large enough to provide a stable indication of 
behavior but be responsive enough to provide for adaptation while the 
learning is still proceeding. 

In summary, this investigation attempted to answer the ^question 
of whether regression analysis techniques can be used to identify the 
within-course variables that related to final performance, and whether 
the decision logic can be built from these techniques to provide dynamic 
adaptive training via computer-assisted instruction. 
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SECTION m 



METHOD 



A, INSTRUCTIONAL MATERIALS 



The mateHals which were used in this study were a portion of a 
Boolean algebra course which had been previously developed for the Naval 
Training Device Center under Contract No. 68-C-0071, These materials 
were specifically selected since the target population for the develop- 
ment and verification for the adaptive decisicm. mode,! was a group of 
Naval Reserve officers and enlisted men. 

The content of the Boolean algebra course which was selected 
for the study included two major sections: introduction to Boolean 
algebra, and three Boolean algebras (mathematical logic, set theory, and 
switching networks). (Appendix A contains a list of the eight concepts 
comprising these two sections.) A task analysis of the concepts in the 
Boolean algebra course indicated that they form what appears to be a 
sequential hierarchy of interrelated skills which the student must learn. 

The original design of these instructional materials was quite 
similar to Crowder's (1959) limited branching instructional strategy which 
has been employed in the field of programmed instruction. That is, infor- 
mation frames and queJ»tion frames were utilized. The question frames had 
multiple choice formats with four alternatives which required specific 
responses from a trainee. If the trainee responded incorrectly, he was 
told that he was incorrect, and at times was given assistance, i.e., cues 
for identifying the correct answer. 

For the purposes of the present study, the instructional matertaLs 
were divided into two forty-minute instructional units. As shown in 
Appendix A, Unit I included three concepts and Unit II covered five con- 
cepts. For each concept a "preview frame" was written. This was a frame 
which was, in fact, a question which attempted to measure whether a trainee 
had alrea^ learned the concept which was to be taught next. A preview 
frame preceded the presentation of each concept and was clearly labeled as 
a pretest question. Similarly, criterion questions were developed for each 
concept. These criterion questions closely paralleled the intent of , pre- 
view questions in that both were designed to measure the achievement on 
the concept. There was a series of information and question frames between 
the preview and criterion frames for each concept. This segment of the 
training materials will be referred to as the acquisition stage — where the 
learning of the concept is acquired, The number of information and question 
frames within each concept is presented in Appendix A. 
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A fraiTie is operationally defined as that amount of material that 
could be displayed on the cathode ray tube screen at one time which 
required an answer or continue response from the trainee^ Two other 
questions were also developed which were parallel to the preview and 
criterion frames. These questions were used in an end-of-unit quiz and 
in a final examination on all of the instructional materials. Therefore, 
for each concept taught, there were four parallel test questions devel- 
oped. These questions were used for the following purposes: (1) as a 

preview or pretest measure which was administered just prior to instruc- 
tion on a concept, (2) as a criterion measure which tested for acquisition 
of a concept immediately after it had been taught, (3) as an end-of-unit 
question which was utilized to test those concepts which had been taught 
during that session, and (4) as a final examination question which was 
administered at the conclusion of the en’^'re learning sequence. The 
learning sequence for the trainees is shown in figure 1. 



Preview 

Fr^ 

Dm t I 



Inf. & 
Question 
frames 






(acquisition 

stage) 



Criterion Frame 



d H-AHl 0~/~0 



Unit 1 Quiz 
J — I (3 items) 




Unit II 
Quiz A 



Final 

Exam 



5 items 



Unit II 0'^ 




Figure 1.— Trainee Learning Sequence 



B. EQUIPMENT 

The instructional materials were presented by means of the IBM 
1500 system located at the Florida State University Computer-Assisted 
Instruction Center. Each instructional terminal consists of a CRT 
(television screen) display as output, and both keyboard and light-^pen 
response modes as trainee input devices. The trainee's performance 
and latency for each response were recorded automatically by the computer. 
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C. EXPERIMENTAL DESIGN 



There were essentially four steps in the execution of the total 
experimental design. After the learning materials had been completely 
coded and prepared for use by the trainees, a test group of public high 
school students was involved in a pilot study in order to check that the 
instructional program was running effectively and to determine that all 
the learning parameters were being measured and recorded correctly. 
Following this pilot study, a group of Naval Reservists studied the 
Boolean algebra CAI program in its initial form in order to collect the 
learning data which would be utilized to build the adaptive decision 
model. The third step was to use the data from the first group of Naval 
Reservists for developing the adaptive decision model and to insert the 
appropriate remedial instruction into each of the learning units. The 
final step involved having a second group of Naval Reservists proceed 
through the Boolean algebra program to which had been added (1) the coding 
in the program logic to monitor the appropriate within-course variables, 
(2) the decision models which would detect those trainees who would need 
additional instruction, and (3) the remedial loops which would optimize 
their final performance. The initial group of Naval Reservists will be 
referred to as the Phase 1 trainees and the second group of Naval 
Reservists, the validation group, will be referred to as the Phase 2 
trai nees . 



Do MEASUREMENT TECHNIQUES AND MODEL BUILDING 

The three basic variables which were used in developing the adap- 
tive decision model were acquisition performance, latency, and subjective 
confidence. These three measures were recorded several times during the 
presentation of each individual concept. Performance, latency, and con- 
fidence were recorded for each preview question, each criterion question, 
and each quiz question. The percentage correct and the latency during 
the acquisition stage (frames that were presented between the preview 
and criterion questions) were also recorded. 

This collection of independent measures was used as the input for 
the multiple regression analyses. The multiple regression technique was 
selected as the basis for building an adaptive model because: 

1) it identifies those variables which account for a signi- 
ficant amount of the performance, and thus can be used 
in the adaptive decision model, and 

2) it produces weighting factors which can be applied to the 
performance data to predict future performance. 

Given the input of variables mentioned above, the ^problem ^remained 
to determine* the unit of analysis to be utilized in building the instruc- 
tional model. Previous studies, in which remedial branching techniques ■ 
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were applied on the basis of the response to a single learning frame, 
have had limited success, it was hypothesized that a larger un'^t or 
sample of behavior was needed in order to provide a stable indication of 
the trainee's performance,. The largest sample of behavior would be per- 
formance across the total course In this case, adaptation would take 
place only after the basic core of training was completed. Since the 
material did seem hierarGhical in nature, the most appropriate unit of 
analysis would be somewhere between a single frame and the total course. 

It was felt that the umt of analysis should p.rovide a stable indication 
of behavior but be responsive enough to provide for adaptation while the 
learning was still proceeding Therefore, the decision was made to 
develop the adaptive decision models and intervene with remedial instruc- 
tion at four locations within the instructional sequence. Remedial 
instruction was located just following instruction on the final concept 
for each unit, ie., following all of the instructional activities for 
each unit but preceding the end-of-unit quizzes Remedial instruction also 



followed the end-of-unit 
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Figure 2 displays the points of remediation. 
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Figure 2. --Points of Remediation 



The dependent variable used as the criterion for the intervention 
decision prior to the quiz was the predicted unit quiz performance. The^ 
dependent variable for the intervention decision following the end-of-unit 
quiz Was the predicted performance on the final examination for the con- 
cepts involved within the particular unit. In essence, data was accutnulated 
over several concepts in order to determine if remedial activities would ^ 
be required, both following the learning sequence for a unit, and following 
quiz activities for a unit. In order to reach these decisions , data from 
the following variables was available for inclusion in the multiple 
regression equations at the end of the learning sequence: performance, 

latency and confidence ratings on both preview and criterion questions; and 
performance (percent correct) and latency during the acquisition stage. 

This data served as input to predict the performance of the Phase 2 
trainees on the end-of-unit tests » Similarly, after the unit quiz, 
cumulative data on performance, latency, and confidence on the quiz 
served as input to predict performance on the corresponding final 
examination questions. 
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The decision rule for providing remedial instruction for the 
Phase 2 trainees prior to and following the quiz for each unit was that 
a Phase 2 trainee's predicted performance on the quiz or final examina- 
tion had to be equal to or greater than the mean performance on these 
tests by the Phase 1 trainees, whose learning data was utilized in 
building the optimization model. For example, if a Phase 2 trainee's 
predicted quiz performance was less than the mean quiz performance 
achieved by the initial group of Naval Reservists, then that trainee 
was branched to the pre or post quiz remedial instruction for the unit. 



E PROCEDURE 

1. PHASE 1 STUDY 



The subjects were eleven Naval Reservists from the Naval 
Training Reserve Center in Tallahassee. The subjects were drawn from 
a population of enlisted men, and two of the subjects indicated that 
they were familiar with simple Venn diagrams. 

The learning sequence for each trainee was essentially as follows: 
after a brief introduction to the CAI system and the instructional terminal 
he was given instruction via the IBM 1500 system on the use of the confi- 
dence scale,. This was followed by a preview question on the first -concept 
which was to be taught. He was then asked to indicate his confidence in 
his performance on that question. The confidence scale is presented in 
Appendi.x; B. Regardless of his performance, either right or wrong, he 
continued into the acquisition stage materials for that concept. The 
sequence was basically linear. At the conclusion of the instruction 
on that concept he was given a criterion or posttest question. He was also 
asked to indicate his confidence in his performance for that question.. He 
then encountered the preview frame for the next concept. 

This sequence continued over three concepts. After he indicated 
his confidence for the criterion question on the final concept for the 
unit, the end-of-unit quiz was administered via the terminal. The test 
consisted of one question for each of the concepts encountered during that- 
instructional session. The trainees were allowed to take a short break 
before proceeding on to Unit II which contained five concept sequences. 
After the second unit, all students were given a piaper and pencil final 
examination consisting of one test item for each concept of the eight con- 
cepts which had been taught in the entire learning program. After the 
final examination, the subjects were given an attitude questionnaire assess 
ing their opinions about CAI presentation of the Boolean materials. A copy 
of this questionnaire is given in Appendix C. 

The data acquired in Phase 1 was utilized to build the decision- 
models which were used in the Phase 2 study. 
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2. PHASE 2 STUDY 



The subjects vjere eleven Naval Reservists drawn from the same pop^. 
ulation of enlisted men from which the Phase 1 subjects were obtained. 

The instructions and procedures used here were identical to those used in 
Phase 1. For this group, however, the adaptive instructional model had 
been coded into the course logic via the programming language. At the 
selected locations, the subjects could receive remedial instructions 
based upon the outcome of the regression equation calculations which 
were dependent upon the trainee’s performance during the instructional 
program. After completion of the instruction, these subjects were queried 
about their opinion of the effectiveness of the remedial instruction 
they received. The questions concerning the remedial instruction are 
given in Appendix D, 



SECTION IV 



RESULTS 



A, PHASE 1 



The following measures were obtained on the preview questions, 
criterion questions and quiz questions: performance, latency, and con- 
fidence ratings. The percent correct and the latency during the acqui- 
sition stage were recorded for each instructional unit. The final 
measure was the score on the final examination. The data on each of 
the twelve variables was summed across the three concepts in Unit 1 
and across the five concepts in Unit 2. The means and standard devia- 
tions for these resultant twelve variables are given in Table 1. 

Tables 2 and 3 contain the intercorrelations of the variables. 
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TABLE 1 

PHASE 1 - MEANS AND STANDARD DEVIATIONS 

UNIT 1 UNIT 2 





Means 


S.D. 


Means 


S.D. 


Preview Performance^ 


1.82 


Io09 


3.64 


1.20 


p 

Preview Latency 


62.28 


25.26 


216.63 


65.19 


3 

Preview Confidence 


9.09 


2.95 


21.73 


3.80 


Cri terion Performance 


2.45 


i.n 


3.64 


1.28 


Criterion, Latency 


32.47 


1 

8.44 


104.84 


36.04 


Criterion Confidence 


13.00 


3.16 


22.64 


4.52 


Acquisition Performance 


77.27 


. 16.18 


81.64 


14.93 


Acquisition Latency 


773. 32 


163.62 


1298.28 


265.75 


Quiz Performance 


2;09 


1.13 


2.27 


1.36 


Quiz Latency 


39.23 


11.03 


267.21 - 


173.72 


Quiz Confidence 


12.82 


2.56 


21.09 


4.39 


Final Exam 


2.36 


1.13 


2.64 


1.36 



parformance = number of correct responses. For Unit 1 , the 
maximum possible is three i for Unit 2 the maximum possible is five. 

This also applies to criterion performance, quiz performance, and final 
exam. 

2 

All latencies are given in seconds. 

^based on a five point scale (1 « low, 5 ® high confidence), the 
maximum possible for Unit 1 is 15 i for Unit 2 the maximum possible is 25. 
This also applies to criterion and quiz confidence. 
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♦Correlations above .735 are significant at p < .01; for a'll others 




♦Correlations above .735 are significant at p < .01; for all others p < .05. 



. . 1 . nFVFi nPMFNT OF THE ADAPTIVE DECISION imEL 

The analysis of the Phase 1 data indicated unstable relation- 
ships between the preview performance, latency and confidence with 
the remaining variables . Therefore, in building the decision mioclel * 
these three variables were not considered. Since intervention for 
remedial instruction could occur before each of the unit quizzes and/ 
or after each of the unit quizzes, four separate stepwise regression 
analyses were conducted. The following five measures were initially 
entered to predict the unit quiz performance: performance, latency, and 
confidence on the criterion questions; percent performance, and latency 
during the acquisition stage. These five dependent measures were 
regressed on quiz performance. However, in developing the decision 
models for applying remedial instruction after the quiz, quiz perfor-^ 
mance, quiz latency, and quiz confidence were regressed on final exami- 
nation performance for the corresponding units. The predictor variables 
accounting for a major portion of the variance as indicated by the 
multiple R for each of the four regression analyses are given in Table 4. 



TABLE 4 



VARIABLES USED IN THE REGRESSION EQUATIONS AND THE 
MULTIPLE R AND R^ FOR THE EQUATIONS BY UNIT 



\ ... V 


Pre-Qui z 


Post Quiz 




Unit 1 


Acquisition Performance 
Acquisition Latency 
R = .6068 
= .3682 


Quiz Performance 
Quiz Confidence 
R = .6506 

R^ = .4233 




Uni t 2 


Acquisition Performance 
Acquisition Latency 
Criterion Latency 
R - .7674 
R2 = 5889 


Quiz Performance 
R = .5637 
R2 = .3178 





The multiple R squared (R^) shown in Table 4 indicated the amount of 
variance of the dependent variable (quiz and final examination per- 
formance) accounted for or predicted by the combination of variables 
in each of the four equations, The stepwise regression equations with 
the amount of variance accounted for with the addition of each variable 
for each of the four analyses along with the significance levels of 
the final equations are given in Appendix E. If the regression equations 
are to be effective, then the dependent variables (performance on the 



unit quiz, and performance on the final examination) shoul<| Njrellable 
measures 0 The internal consistency reliability estimates qf tpe qwiz 
and final examination for each of the two units are given ip Table o. 



TABLE 5 



PHASE 1 QUIZ AND FINAL EXAMINATION 
RELIABILITIES BY UNIT 




Unit 1 


Unit 2 




QUIZ 


.387 


.484 




FINAL EXAM 


.543 


.539 ^ 





2. CQURSEMRITER 11 IMPLEMENTATION 

The variables used in the regression 'equation for predicting 
quiz performance on Unit 1 (as shown in Table 4) were acquisition per- 
formance and acquisition latency across the ^ three concepts » The reqoding 
of the Coursewriter programming statements involved summing these two 
measures’ across the three concepts in Unit 1, and coding in the egression 
equation calculations immediately after the last^crlterion question In 
that unit. -These modifications permitted a prediction of^how a trainee 
would perform on the quiz prior to his taking the quiz. If his predictea 
Derformance was equal to or greater than the mean performance on the 
quiz for Unit 1 based on Phase 1 data, then the trainee would be branched 
immediately 1;o the quiz. If his performance was less than the meart per- 
formance when compared to the Phase 1 group, he would receive the remedial 
instruction. I" Prior to receiving the remedial instruction, ® trainee s 
latency' and percent performance within Unit 1 was compai^d with the means 
on the comparable variables obtained from the Phase 1 stu^.. The trail nee 
received the' appropriate Instruction indicating whether h’.s performance 
was too low, whether he was taking too much time, or both.. After tn.is pe 
was given the remedial Instruction. . <■ 

The relevant variables in the regression equation for predicting, 
final examination performance on the first three concepts in Uni t' 1 
were quiz performance and quiz confidence. The recoding of pe Course* 
writer II statements involved summing the quiz performance and quiz 
confidence across the three quiz questions from Unit «• 
equation calculations were coded into the computer logic. so that tiie 
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equations were calculated immediately after the last quiz question. 

For each trainee then, a prediction of his final examination performance 
on Unit 1 was obtained before he took the final examination. If his 
predicted final examination performance was less than the mean final 
examination performance on Unit 1 from the Phase 1 group, he was given 
remedial instruction Prior to receiving the Unit 1 post quiz remedial 
instruction, the trainee received special instructions informing him 
why he was receiving the review. The same process was followed for 
Unit 2 using the variables as Indicated in table 4. 



3. DEVELOPMENT OF THE REMEDIAL MATERIALS 

An item analysis of the criterion, quiz, and final examination 
questions was conducted for the pilot study trainees and the Phase 1 
trainees. Analysis of these questions indicated certain learning 
problems within each concept. The remedial materials were specifically 
developed to alleviate these problems. The remedial materials which 
were inserted consisted of a review of each concept presented followed 
by one or two questions. Each of the questions was response sensitive 
in that each time a wrong alternative was selected, special feedback 
was given which indicated why it was wrong and a hint as to why the 
misconception might have arisen. The Unit 1 pre-quiz remediation con- 
tained three information frames and three response sensitive question 
frames. A parallel set of remedial materials was developed for the 
Unit 1 post-quiz adaptation Unit 2 remediation consisted of two 
parallel sets of five information frames and six response sensitive 
question frames. 



B. PHASE 2: THE VALIDATION STUDY 

The variables used in the analysis of the Phase 2 Naval Reservists 
data were the same 12 variables identified in the Phase 1 study. The. 
means and standard deviations per concept of these 12 variables for each 
of the 2 units are given in Table 6. Table 7 shows the number of times 
remedial instruction occurred for each trainee, and the amount of time, 
in seconds, that he spent on this instruction. A coding error kept 
all trainees from receiving the Unit 2 post-quiz remedial instruction. 

A post hoc analysis of the regression equation calculations indicated 
that 6 of the 11 trainees should have received remedial instruction. 

The mean latency during acquisition for the seven trainees who 
received remedial instruction in Unit 1 was 951,1 seconds. The mean 
time taken to examine and respond to the remedial instruction was 
229.2. The additional time required for remedial instruction represents 
a 24 percent increase in time to complete the Unit. For Unit 2, the 
mean latency during acquisition for the nine trainees who received 
remedial instruction was 1934.2 seconds. The mean time taken by these 
nine trainees on the remedial instruction was 229,2 seconds. The addi- 
tional time required for remedial instruction in Unit 2 represents a 
22 percent increase in time. 
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TABLE 6 



PHASE 2 - PERFORMANCE, LATENCY, AND CONFIDENCE: 
MEANS AND STANDARD DEVIATIONS 







UNIT 1 




UNIT 2 


1 

Preview Performance 


Means 


S.D. 


Means 


S.D. 


1.09 


.90 


3.36 


.87 


Preview Latency^ 


56.77 


25.34 


223.98 


50.04 


3 

Preview Confidence 


8.82 


2.75 


20.00 


4.90 


Criterion Performance 


1.55 


.79 


4.00 


.81 


Criterion Latency 


47.53 


25.57 


97.95 


23.36 


Criterion Confidence 


ii;9i 


2.55 


21.64 


5.22 


‘Acquisition Performance 


71.82 


21.83 


73.73 


11.83 


Acquisition Latency 


812.05 


181.19 


1502.84 


403.32 


Quiz Performance 


2.00 


1 ,00 


1.73 


1.35 


Quiz Latency 


46.15 


17.91 


259.54 


149.23 


Quiz Confidence 


11,82 


2.64 


18.45 


4.80 


Final Exam 


2,18 


.75 


2.45 


1.04 



^performance ® number of correct responses. For Unit 1, the 
maxi ml m possible Is three; for Unit 2 the maximum possible Is five. 

This also applies to criterion performance, quiz performance, and 
final exam. 

^All latencies are given In seconds. 

3pased on a five point scale (1 * low, 5 = high confidence), the 
maximum possible for Unit 1 Is 15; for Unit 2 the maximum possible Is 
25. This also applies to criterion and quiz confidence-. 



TRAINEES RECEIVING REMEDIAL INSTRUCTION 
AND THEIR MEAN LATENCY BY UNIT 
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C. COMPARATIVE ANALYSIS OF PHASE r AND PHASE 2 

It was felt that due to a series of methodological problems and 
the small number of subjects, it would be inappropriate to conduct a 
series of sophisticated statistical analyses in comparing the control and 
experimental groups. It is often the case that -such analyses lend undue 
authenticity to findings that should not betakenas conclusive, conse- 
quently limitations of the studies are not given proper consideration. It 
should be pointed out that this particular-approach to adaptive training 
was an initial developmental effort. Therefore, the comparative results 
are presented for descriptive purposes only. The data for the following 
graphs are taken from tables 1 and 6. Overall', the training material used 
in this study was an introduction to Boolean algebra. The data from each 
unit are presented separately since the two units differed in basic content 
as well as degree of difficulty. 



PERFORMANCE 

Figures 3 and 4 present the mean'humber correct on the preview, 
criterion, quiz and final examination questions for Unit I and Unit II 
respectively. In making comparisons between Unit I and Unit II it should 
be remembered that Unit I reflects performance oh three concepts whereas 
Unit II reflects performance on five concepts. It should also be noted 
that the preview and criterion questions were spaced throughout the instruc- 
tion for each unit, the quiz questions were presented in a block at the 
end of each unit, and the final examination questions were presented after 
all instruction was completed. 

The points of adaptation in terms of remedial instruction took 
place before and after the quiz for each unit. Therefore, the major com- 
parisons of interest are performance on the quiz and final examination 
questions. Figure 3 shows that the adaptive model group (Phase 1) started 
with less prior knowledge and gained slightly less in comparison with the 
linear training group as indicated by performance from preview to criterion. 
However, the performance of the adaptive model 'group (Phase 2) on the 
Unit I quiz and final examination questions was comparable to that of the 
control group (Phase 1). In addition, the control group shows a drop in 
performance from the criterion to the quiz whereas the adaptive model 
group shows a gain. 

Figure 4 indicates that the apparent difference in prior knowledge 
of the two groups shown in figure 3 is no longer noticeable in Unit II. 

The increased proficiency of the adaptive model -group (Phase 2) in Unit I 
was maintained in Unit II. Figure 4 also 'indicates ’ that the performance 
on all measures in Unit II for the two groups was quite parallel. The 
drop in performance from the criterion to -the' quiz' and final examinatiort 
questions indicates that either the training materials in Unit II were- 
poorly designed or that the quiz and final examination were much more dif- 
ficult than the criterion questions. Apparently 'the' remedial training 
presented in this unit was not sufficient to overcome either case. 

20 



o 



Mean Number Correct 




Figure--Mean Number Correct on Unit 1 Preview, 

Criterion, Quiz and Final Examination Questions 
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Figure 4==Mean Nuipber Correct on Unit II Preview, 

Criterion, Quiz, and Final Examination QiJiestions 



22 



2 LATENCY 



Figures 5 and 6 present the mean latency in seconds on the preview, 
criterion and quiz questions for Unit I and Unit II respectively. The 
time taken on the preview and quiz questions in Unit 1 for the two groups 
was fairly similar. The larger difference in time taken between the two 
groups on the criterion questions may be a reflection of the difference 
in performance on this measure as was shown in figure 3. That is, the 
increased time taken by the adaptive model group (Phase 2) on the criterion 
questions in Unit 1 may be a reflection of the Increased difficulty of 
these items as shown in figure 3. As indicated in figure 6, the latency 
measures in Unit II for the two groups are almost overlapping. The great 
increase in time taken on the quiz questions for both groups as compared 
with the criterion questions parallels the finding of low quiz performance 
as compared with criterion performance shown in figure 4, This increased 
time taken on the quiz in Unit II would seem to indicate that the questions 
were difficult and further that the trainees were attempting to answer 
the question rather than merely guessing. 



3. CONFIDENCE 



Figures 7 and 8 present the mean confidence of the trainees in 
their responses on the preview, criterion, and quiz questions for Unit I 
and Unit II respectively. It should be noted in figure 7 that confidence 
for both groups in Unit I was low on the preview questions, as would b© 
expected. Confidence increased after instruction (I.e., on the criterion 
questions), and was maintained on the quiz The confidence is para^el 
across the three measures for the two groups. The confidence on Unit II 
as seen in figure 8 is again relatively parallel for the two groups. On 
Unit II however, confidence is rather high on the preview measures,^ 
increases only slightly after instruction, and then drops on the quiz 
questions. This would appear to indicate that the quiz items were not 
measuring the same thing as were the preview and criterion questions 
(i.e., they were not parallel) as was Indicated by the performance measures 
on this unit. 



4. ATTITUDES 

A sum score on the attitude questionnaire for each subject was 
obtained by scoring a / for strongly disagree to a 5 for strongly agree 
on each positive item and the reverse for negative Items, The means 
and standard deviations for each group are given in table 8. 
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TABLE 8 



MEAN ATTITUDE SCORE FOR PHASE 1 AND PHASE 2 TRAINEES 







MEANS 


STANDARD DEVIATION 


Phase 1 




50.45 


4.51 


Phase 2 




49.09 


6.04 



A t-test for significance of the difference between the means 
yielded a t of .62 which, with 20 degrees of freedom, is not signifi- 

cant at the .05 level . 
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SECTION V 



DISCUSSION 



A, REVIEW OF THE RESULTS 



The results on Unit I of the training materials present a generally 
favorable picture fo>" the adaptive model group. The pre-quiz decision 
model directed three of the eleven trainees to remedial instruction. Of 
these three, one was also directed to remediation after the quiz. A 
total of five trainees received remediation after the quiz (see table 7). 
The performance measures indicated that although the adaptive model group 
had a lower entry level of behavior and gained slightly less during 
instruction, they achieved about the same level of performance as the 
control group (Phase 1) on the quiz and final examination for the unit 
(see figure 3). It should be pointed out that the decision criterion for 
branching a trainee to remediation was that his predicted performance had 
to be equal to or greater than the mean of the control group. In this 
sense then, the adaptive decision model was effective. Had a more 
stringent criterion level been used, the overall performance of the 
adaptive model group might have been raised even higher. However, since 
performance on the quiz was at the 50 percent level and performance on 
the final examination was only slightly higher, it might also be posited 
that if the remedial instruction had been more extensive, the adaptive 
model group would have performed even better on the quiz and final 
examination. The relative effect of the cv'iterion level and the 
extensiveness of the remediation is difficult to assess given the 
relatively small number of subjects in this investigation. It had been 
anticipated that 20 or more trainees could be obtained for the Phase 1 
study; however, when the time came for the study to be conducted many 
of the Naval Reservists had conflicting duties 

The data on Unit II, on the other hand, did not show such a 
favorable trend for the adaptive model group. This finding appears to 
be more a function of the design of the training materials than of 
the adaptive model. The performance, latency, and confidence measures 
in Unit II (see figures 4, 6, and 8) for the adaptive model group and 
the control group are almost overlapping. The extreme drop in perform- 
ance on the quiz and final examination in Unit II for both groups (as 
shown in figure 4) indicated that the quiz and final examination ques- 
tions were not comparable with the preview and criterion items. Indeed, 
they appear to be more difficult. If it had just been the case that 
the training materials were poorly designed, such an extensive drop 
should not have occurred. Nine of the eleven trainees had been directed 
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to the Unit II pre-quiz remediation, but this additional instruction did 
not seem to help them on the quiz. The trainees in both groups took 
more than twice the amount of time to answer the quiz questions than 
they did on the criterion questions (see figure 6). Six of the nine 
trainees who received the pre-quiz remediation should have received the 
post-quiz remediation but did not due to a coding error in the computer 
language. 

It is difficult to assess the effect of this error particularly 
in light of the apparent difference in difficulty of the quiz and final 
examination questions as compared with the criterion questions. It is 
interesting to note that the six trainees who missed this instruction 
had a mean score of 1.83 on the five final examination questions covering 
that unit, whereas the remaining five trainees who did not need ^ this 
Instruction had .a mean score of 3.20 on the five final examination ques- 
tions covering Unit II. 

The lack of parallel development of the quiz and final examination 
questions with the preview and criterion questions in Unit II is also 
Implied in the findings on the confidence measures (see figure^S). Confi- 
dence is rather high on the preview questions and increases slightly 
on the criterion questions. However, the trainees confidence on the ^ quiz 
drops below that on the preview items. This was not the case for Unit I 
(see figure 7). Confidence was low on the preview, increased on the 
criterion questions, and was maintained at about the same high level on 
the quiz for both groups. 



B. METHODOLOGICAL CONSIDERATIONS 

Several methodological procedures employed in this investigation 
that might have hindered the demonstration of extent of^the 

effectiveness of the adaptive procedures employed should be mentioned. 

The first of these relates to the preview items. These items were 
spaced throughout the instruction so that if performance on these measures 
did relate to- final performance, a trainee might be branched ahead to 
the next concept if he answered that preview question correctly. These 
measures did not show any significant relationship with final ^performance, 
and they were not used in the model building. It did appear in Unit I 
that the adaptive model group did have less prior knowledge. ^ However, 
since the preview questions were spaced out over the instruction, and 
since tbe training materials did appear to be heirarchical in nature, it 
would have been difficult to determine the groups' true entry level to 
effectively show relative gains. Perhaps a complete pretest should have 
been given before any training started. 

Another hinderance relates to the nature of the evaluation 
items... It should be recalled that one question was used to assess 
mastery of a concept, and a parallel question was developed for the end- 
of -uni t quiz in addition to one for the final examination. A salient 
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question in educational research is what are the appropriate criteria 
for determining whether a learner has mastered a given concept. Is one 
question enough? Are two questions better than one, or three better 
than two? As indicated in table 5, the quiz and final examination 
reliabilities for the Phase 1 data were not extremely high. This would 
indicate that the criteria for assessing mastery were not as effective 
as they should have been. Considering the degree of difficulty of the 
Unit II instructional materials, one question per concept on the criterion, 
quiz, and final examination measures may not have given an effective 
measure of mastery of the concept. 

A further consideration relates to the variables that were 
identified to be measured during the learning process for possible inclusion 
in the adaptive model. Performance and latency were measured on preview, 
acquisition, criterion and quiz frames. Confidence was assessed on pre- 
view, criterion and quiz questions. Table 4 presented the variables 
that were selected for inclusion in the regression equations based on 
the Phase 1 data.., Considering the discussions relating to the difficulty 
of the material, the nature of the evaluation items, and the sample size, 
the question arises as to whether the variables identified were the most 
effective ones and whether other variables might have been selected. 

Those variables that involved the largest number of observations, i.e., 
acquisition performance and latency, seemed more reliable. The relation- 
ships of the confidence measures to the other indices were somewhat 
unstable. The difficulty of the materials may well have contributed to 
these unstable relationships. However, a more effective assessment of 
a trainee's confidence might have been to simply ask him if he felt 
ready to proceed to the next concept. 

The final consideration relates to the procedure of telling 
the trainee that he was receiving remedial instruction and why. The 
rationale was that if the trainee knew why he was performing below 
average, he would be able to effectively improve his performance,. How- 
ever, this might have had a deleterious effect in terms of reducing the 
trainee's motivation, particularly if he felt that the instruction was 
difficult and irrelevant. 
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SECTION VI 



CONCLUSION 



A methodology for producing a model for adaptive training via 
computer- ass is ted instruction utilizing regression analysis techniques 
was developed. The model was shown to be effective in identifying 
those trainees who needed remedial instruction. The results showed 
that although the adaptive model group started with less prior know- 
ledge and performed at a lower level throughout the mainline of training, 
they performed comparably on the final examination as compared with a 
group that studied what was essentially a linear program. This result 
was only seen on the first half of the training materials. Certain 
methodological considerations such as the difficulty of the material, 
the nature of the remedial instruction, the development and number of 
the evaluation items and a coding error in the computer programming 
language were discussed as possible inhibiting factors in demonstrating 
the true effectiveness of the adaptive model on the second half of the 
training materials. 
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SECTION VII 
RECOMMENDATIONS 



The Implications for future research in the development and 
utilization of adaptive models in computer-assisted instruction 
are these: 

1. Regression analysis techniques can be utilized to identify 
those variables that relate to final performance and to 
establish the decision logic to be employed in the develop- 
ment of adaptive training models- These techniques could 
have high payoffs where the training system allows for 
real-time analysis of each individual's performance. 

2. In developing the adaptive training models the instructional 
materials should be difficult enough so that some variance 
in performance is allowed to exist in order to establish 
relationships among the variables, but not so difficult 

as to create an excessive amount of variance. 

3. Further research needs to be conducted to establish the 
appropriate criteria for determining whether a learner 
has mastered a given concept and to establish the 
appropriate type and amounts of instruction to be pre- 
sented after determination of a learner's lack of 
mastery . 

4. Further investigations should be conducted to identify 
other variables that might be effectively utilized 

i n adapt! ve tral ni ng model s o 



In this investigation a large step program with remedial 
instruction was used. Consideration should be given ^to the use^of 
other program designs such as a small step program with a decision 
logic employed to branch trainees ahead if they are performing 
satisfactorily. In addition, adaptive training models might well 
be utilized in Computer-Managed Instruction where the majority of 
the trainee's time on the computer is devoted to evaluation of 
performance, and the bulk of instruction presented "off-line. 
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APPENDIX A 

I NS TRU CTIQNAL MATERIALS 



After all revisions, additions, and deletions the final Boolean 
algebra program included two Units and eight concepts as shown below. 
The-nyrober of information and question frames is given by conceptand 

unit. 



o 

ERIC 



Unit 1 



(Concept units) 

1. Elements in mathematical logi-c 

2. Elements in set theory 

3. Elements in switching networks 

TOTAL FRAMES 



Unit II 



(Concept units) 

4. AND operation in logic 

5 . and operation in set theory 

5 . and operation in switching networks 

7. OR operation in logic 

8. OR operation in set theory 

TOTAL FRAMES 
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INFORMATION 

FRAMES 


QUESTION 

FRAMES 


8 *• 


3 


8 


3 


9 


4 


25 


10 


INFORMATION 

FRAMES 


QUESTION 

FRAMES 


6 


2 


6 


12 


4 


5 


5 


3 


6 


13 


27 


36 
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APPENDIX B 

nONFIDENCE RATING SCALE 



1 I GUESSED AT THE ANSWER. 

2 I KNEW A LITTLE ABOUT THE QUESTION. BUT MY 
ANSWER IS PROBABLY WRONG. 

3 I AM HALF SURE MY ANSWER IS CORRECT. 

4 I AM PRETTY SURE MY ANSWER IS CORRECT. 

5 I AM SURE MY ANSWER IS CORRECT. 
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APPENDIX C 



ATTITUDE QUESTIONNAIRE 



(NOTE: The Phase 1 summary data are given above the scale for each of 
the questions and the Phase 2 summary data are given below the scale 
for each of the questions.) 

This is not a test of information, therefore, there is no one 
“right" answer to a question. Your opinions will be strictly xonfi- 
dential. You do not need to put your name on this questionnaire. Please 
be frank and honest in your replies. 



Thank youl 



As a change of pace from usual 


training activities, Computer-Assisted 


Instruction was 


wel come 










• 


• 


7 

• 


4 

• 


• 

Strongly 


• 

Disagree 


• 

Uncertain 


• 

Agree 


• 

Strongly 


disagree 








agree 


1 


1 


7 


2 


I would like to 


have more of my 


Naval Reserve 


trai ni ng 


by Computer- 


Assisted Instruction. 












1 


7 

• 


3 

• 


• 

Strongly 


» 

Disagree 


Uncertain 


• 

Agree 


• 

Strongly 


disagree 








agree 


1 


3 


5 


2 



3. I enjoyed working at my own pace through the materials 

5 



Strongly 

disagree 



Disagree Uncertain Agree 



6 

• 

Strongly 

agree 

6 



5 



4. I felt as if I had a private tutor while on Computer-Assisted Instruc- 
ti on . 



2 

: : 
Strongly Disagree 

disagree 



1 


5 

• 


3 

• 


• 

Uncertain 


« 

Agree 


• 

Strongly 






agree 


3 


3 


4 



While taking Computer-Assisted Instruction I 
best work. 


felt uiallenged to do my 




1 2 


6 

• 


2 

• 


• 

• 

Strongly 

disagree 


« • 

• • 

Disagree Uncertain 

2 


• 

Agree 

5 


• 

Strongly 

agree 

4 


I was encouraged by the computer's immediate 


response to 


my questions. 




1 


6 

• 


4 

• 


• 

• 

Strongly 

disagree 


• • 

» • 

Disagree Uncertain 

2 


• 

Agree 

5 


• 

Strongly 

agree 

4 



(NEGATIVE ITEM) . . 

7. The Computer-Assisted Instruction situation made 



me feel quite tense. 



• 

• 

Strongly 


7 

• 

• 

Disagree 


1 

* 

Uncertain 


disagree 

1 


4 


1 



3 

Agree 

1 



Strongly 

agree 

3 



(NEGATIVE ITEM) 

8. I found myself just trying 
trying to learnc 


to get through the 


material 


rather than 


1 


7 


2 

• 


1 

• 


• 


• 

#» 

Strongly 

disagree 

3 


• 

• 

Disagree 

3 


• 

Uncertain 

3' 


• 

Agree 

2 


Strongly 

agree 
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(NEGATIVE ITEM) 

9. I felt that the course was somewhat difficult. 



• 


4 

• 


3 

• 


4 

• 


• 


• 

strongly 

disagree 


Disagree 

6 


• 

Uncertain 

r 


• 

Agree 

4 


• 

Strongly 

agree 


(NEGATIVE ITEM) 
10. I was given 


answers but I still 


did not understand the 


questions . 


1 

• 


6 

• 


2 

• 


2 

• 


• 


• 

Strongly 

disagree 

1 


r> 

Disagree 

5 


• 

Uncertain 

4 


• 

Agree 

1 


• 

Strongly 

agree 


11. In view of the effort I put into it, I was satisfied with what I 
learned while taking Computer-Assisted Instruction. 


• 

• 

Strongly 

disagree 


• 

• 

Disagree 


r 

• 

Uncertain 

2 


’ 9 

• 

• 

Agree 

6 


1 

• 

• 

Strongly 

agree 

3 


(NEGATIVE ITEM) 
12. While taking 


Computer-Assisted 


Instruction 


I felt isolated and alone. 


1 

• 


9 

• 


• 


1 

• 


• 


• 

Strongly 

disagree 


• 

Disagree 


• 

Uncertain 


• 

Agree 


• 

Strongly 

agree 



2 3 2 3 1 



13. The computer could be widely used in n\y Naval Reserve training. 



9 
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2 

• 


4 

• 


5 

• 


• 

Strohgly 


• 

Disagree 


• 

Uncertain 


• 

Agree 


• 

Strongly 


disagree 








agree 






2 


8 


1 



14. In your opinion, what was the most desirable feature of the Computer'- 
Assisted presentation? 

Phase 1— ability to proceed at own pace (8); wrong answers were 
explained (2); immediate feedback (1) 
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Phase 2~~can proceed at own pace (5); Immediate feedback (4); 

revi ew of materl a1 (1 ) ; 1 1 di dn * t outrank me (1 ) 

15. In your opinion, what was the most undesirable feature of the Com- 
puter-Assisted presentation? 

Phase 1— no review allowed (3); brightness and jumping of figures 
(3). Phase 2— impersonal (1); too quick reaction (1); fuzzing 
and jumping diagrams (2); tendency to guess at materials you don't 
understand or like (1). 



16 . If you have any other comments please make them in the space- below. 



APPENDIX D 



TR AINEES' COMMENT SHEET 



Please indicate below whether you received any of the reviews before 
and/or after either quiz.. Also indicate your opinion of the state- 
ments you received before each review and whether you felt that the 
reviews were helpful- 

Did you receive the review before quiz 1 ? What was your reaction 

to the statement you received before the review? 

^ DTT'yb'OeeT~l.hat the review was effective? 
If not, why not? 



Did you receive the review afte r quiz 1 ? What was your reaction 

to the statement you received before the "revi ew? 

_ review was effective^ _ 

If not, why not?"__ 



Did you receive the review b efo re quiz 2 ? What was your reac 

tion to the statement you received before the review? 

. Did 'you feel that the review was effective? 

If not, why not? 



Did you receive the review after quiz 2 ? What was your reaction 

to the statement you received before the review? 

Did you feelThaTlEhe review was effective? 

If not, why not? 
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APPENDIX E 



STEPWISE REGRESSION EQUATION CALCULATIONS FOR EACH 
OF THE FOUR INTERVENTION POINTS 






UNIT 1 PRE-QUIZ 
Step I 

Percent Performance = Multiple R = .5677, R 
P “ .03958 (x^) - 1.60417 
Step 2 



2 > 



.3223 



Percent Performance = x.j Learning Latency = x^ 



Multiple R = .6068, R^ = .3682 



.10 < p < .25 



y = .3397 (xj + .00158 (xg) - 2.39090 






UNIT 1 POST-QUIZ 
Step 1 

2 

Quiz Performance = x^ Multiple R = .493s R ® 



.2429 



y = .49286 (x ) + 1.82857 



Step .2 ■ 

Quiz (performance = Quiz tojifideno 



:e = x. 



Multiple R - .6506, R^ = .4233'' .10 < p <.25 



y ■ .43487 (x ) + V8^78 (x ) 

2 



.50693 
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UNIT 2 PRE-QUIZ 
Step 1 

2 

Percent Perforinance ~ Multiple R = 3.712, R ® .1378 
y “ .03386 (x^) - .40042 
Step 2 

Percent Performance = x Criterion Latency ~ x« 

1 *- 

2 

Multiple R -• .5551 , R = .3081 
,03870 (x^) t ,01573 (x^) - 2.44452 

Step 3 

Percent Performance - x^ Criterion Latency = X 2 
Learning Latency = x^ 

Multiple R = .7674. = -5889 .05 < p < .10 

y .09705 (x ) ^ .05823 (x^) - .00682 (x^) - 2.80892 

UNIT 2 POST-QUIZ 
Step 1 

Quiz Performance = x 

1 

Multiple R ^ 5637, R^ =.3178 .05 < p < .10 

y =•■ .5673 (x^) + 1.30392 
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